A General Class of No-Regret Learning Algorithms and Game-Theoretic Equilibria

A general class of no-regret learning algorithms, called no-Φ-regret learning algorithms, is defined which spans the spectrum from no-external-regret learning to no-internal-regret learning and beyond. The set Φ describes the set of strategies to which the play of a given learning algorithm is compared. A learning algorithm satisfies no-Φ-regret if no regret is experienced for playing as the algorithm prescribes, rather than playing according to any of the strategies that arise by transforming the algorithm’s play via the elements of Φ. We establish the existence of no-Φ-regret algorithms for the class of linear transformations. Analogously, a class of game-theoretic equilibria, called Φ-equilibria, is defined, and it is shown that the empirical distribution of play of no-Φ-regret algorithms converges to the set of Φ-equilibria. A necessary condition for convergence to this set is also derived. Perhaps surprisingly, for the class of linear transformations, the strongest form of no-Φ-regret learning is no-internal-regret learning. Hence, the tightest game-theoretic solution concept to which such no-Φ-regret algorithms converge is correlated equilibrium. In particular, Nash equilibrium is not a necessary outcome of learning via such no-Φ-regret algorithms.